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(57) Methods and apparatus for hosting a network 
service on a cluster of servers, each including a primary 
and a secondary Internet Protocol (IP) address A com- 
mon cluster address is assigned as the secondary ad- 
dress to each ol the servers in the cluster. The cluster 
address may be assigned in UNIX-based servers using 
an ifccntig alias option, and may be a ghost IP address 
that ts not used as a primary address by any server in 
the cluster. Client requests directed to the duster ad- 
dress are dispatched such that onV one of the servers 
ol the cluster responds to a given client request The 
dispatching may use a routing-based technique, in 



which all client requests directed lo the cluster addrass 
are routed to a dispatcher connected to the local net- 
work ol the server cluster. The dispatcher then applies 
a hash function lo Ihe client IP address n order to select 
one of the servers to process the request. The dispatch- 
•ng may alternatively use a broadcast -based technique 
in which a router broadcasts client requests having the 
cluster address to ail of the servers of the cluster over 
a local nei work. The servers then each provide a filtering 
rcutino. which may involve comparing a server Identifier 
wuh a hash value generated from a client address in 
order to ensure that only one server responds to each 
request broadcast by the router. 
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Description 

Field of the Invention 

The present invention relates generally to data 
communication networks such as the Internet and more 
particularly to lechniques tor hosting network services 
on a cluster ot servers used to deliver data over a net- 
work In response to client requests, where the cluster of 
servers can be collectively identified by a client using a 
single-address image. 

Background of the Invention 

With the explosive growth ot the World Wide Web, 
many popular Internet web sites are hoavily loaded with 
client requests. For example, it has been reported in S. 
L Garfinkel. The Wizard of Netscape," Webserver 
Magazria. July/August 1996, pp. 59-63. that homepag- 
es ot Netscape Communications receive more than 80 
million client requests or "hits* per day, A single server 
hosting a service Is usually not sufficient to handlo this 
type of aggressive growth. As a result, clients may ex- 
perience stow response times and may be unable loac- 
cess certain web sites. Upgrading the servers to more 
powerful machines may not always be cost-effective. 
Another common approach involves deploying a set of 
machines, also known as a cluster, and configuring the 
machines to work together to host a single service. Such 
a server cluster should prderabty publicize only one 
server name for the entire cluster so that any configura- 
tion change inside the cluster does not affect client ap- 
plications. The WorW Wide Web and other portions of 
the Internet utilize an application-level protocol, known 
as tho Hyportoxl Transfer Protocol (HTTP), which is 
based on a dient/server architecture. The HTTP proto- 
col is deserted in greater detail in •Hypertext Transfer 
Protocol - HTTP/t.0.' Network Working Group, May 
1996. <hnp://www.ics.uci.edu/pub/ielf/hltp>. which is 
incorporated by reference herein. 

FIG. i illustrates an exemplary client/server archi- 
tecture suitable for implementing HTTP-based network 
services on the Internet. A client 1 2 generates an HTTP 
requesi for a particular service, such as a request tor 
information associated with a particular wob she. and a 
Transmission Control Protoccflnternet Protocol (TCP; 
IP) connection is then established between the client 1 2 
and a server 14 hosting the service. The client request 
is delivered to the server 14 in this example via a TCP/ 
IP connection over a first network 16, a router IB and a 
second network 20. The Iir6i network 16 may be a wide 
area communication network such as the Inlernel. while 
the second network 20 may be an Ethernet or other type 
ct local area network (LAN) interconnecting server 1 4 
with other servers in a server cluster. The router 16, also 
referred to as a gateway, performs a relaying function 
Between the lirst and second not works which is trans- 
patent to the client 12. 



The client requesi is generated by a web browser 
or other application-layer program operating in an appli- 
cation layer 22-1 of the client 12, and Is responded to 
by a file transfer system or other program in an applica- 

s ,lion layer 22-2 of the server 14. The requested network 
service may be designated by a Uniform Resource Lo- 
cator (URL) which includes a domain name identifying 
the server 14 or a corresponding server cluster hosting 
the service. The a pp Ileal Ion-lev el program ol the client 

"> 1 2 initiates the TCP/IP connection by requesting a local 
or remote Domain Name Service (DNS] to map the serv- 
erdomain name loan IP address. The TCP and IP pack- 
et routing functions in client 12 and server 14 are pro- 
vided in respective TCP layers 24-1, 24-2 and IP layers 

« 26-1 . 26-2. The TCP and IP layers are generally asso- 
ciated with 1he transport and network layers, respective- 
ly, of the well-known Open Systems Interconnection 
(OS!) model. The TCP layers 24-1, 24-2 process TCP 
packets of the client request and server response. Tho 

20 TCP packets each include a TCP header identifying a 
port number ot tie. TCP connection between the client 
12 and 6orver 14. The IP layers 25-1. 26-2 process tP 
packets formed from the TCP packets ol the TCP layers. 
The IP packets each include an IP header identifying an 

25 IP address of the TCP/l P connection between the client 

12 and server 14. 

The IP address for a given network service may be 
determined, as noted above, by the client accessing a 
conventional DNS. The IP layer 26-1 ot the client 1 2 us- 

30 es the resulting IP address as a destination address in 
the IP packet headers of client request packets. The IP 
address together with the TCP port number provide the 
complete transport address for the HTTP server proc- 
ess. The client 12 and server 14 atso Include data link 

35 and physical layers 28-1 lor performing framing and oth- 
er operations to configure client request or reply packets 
for transmission over the networks 1 6 and 20. The router 

13 includes data link and physical tayors 28-3 lor con- 
verting dent request and server reply packets to IP 1or- 

40 met. and an IP layer 26-3 tor performing packet routing 
based on IP addresses. The server 14 responds to a 
given cl ; en 4 , request by supplying the requested informa- 
tion over the established TCP/IP connection in a number 
of reply packets. The TCP/IP connection is then closed. 

<s ThBre are many known techniques for disU touting 
HTTP client requests to a cluster ot servers. FIGS. 2 
and 3 illustrate server-side single-lP-eddress Imago ap*. 
preaches which present a single IP address to the cli- 
ents. An example ot this approach is the TCP router ap* 

so proach described in D.M. Dias. W. Kish, a Mukherjee 
and R Tewari, 'A Scalable and Highly Available Web 
Server," Proceedings ol COMPCON '96, pp. 85-92. 
1996. which Is incorporated by reference herein. FIG. 2 
illustrates the TCP router approach in which a client 12 

ss establishes a TCP/IP connection over Internet 30 with 
a server-side router 32 having an IP address RA The 
routor 32 is connected via a LAN 36 to a server dust or 
34 including N servers 14-i, i= 1,2. ... N, having res pee- 



2 



EP 0 865 180 A2 



live IP addresses 31 , S2. ... SN. Each server of the clus- 
ter 34 generally provide* access to the same set of con- 
tents, and the contents may be ropUcated on a local disk 
of each server, shared on a network file system, or 
served by a distributed file system. 

The sfngle-eddress image is achieved by publiciz- 
tng the address RA of the server-side router 32 to the 
dients via the DNS, The client 12 therefore use* RA as 
a destination IP address In lis request. The request is 
directed to the router 32, which then dispatches the re- 
quest to a seleeled server 14-k of server cluster 34 
based on load charact eristics, as Indicated by the 
dashed line connecting diem 1 2 to server 1 4-k via rouier 
32. The router 32 performs this dispatching function by 
changing the destination IP address of each incoming 
IP packet of a gfvon client request from the router ad- 
dress RA to the address Sk of selected server 14-k. The 
selected server 14-k responds to the client request by 
sending reply packets over the established TCP/IP con- 
nection, as Indicated by the dashed line connecting 
server 14-k to orient 12. In order to make the TCPyIP 
connection appear seamless to the client 1 2. the select- 
ed server 1 4-k changes the source IP address ri it s reply 
packets from rts address Sk to the rouier address RA. 
Th e advantages of this approach are that It does not in- 
crease the number ol TCP connections, and it is totaBy 
transparent to the clients. However, since the above- 
noted source IP address change Is performed at the IP 
layer in a given server the kernel coda of every server 
tn the cluster has to be modified to implement this mech- 
antsm A proposed hybrid of the DNS approach and the 
TCP router approach, in which a DNS server selects ono 
of several clusters of servers using a round-robin tech- 
nique, surfers from the same problem. 

FIG. 3 illustrates a server-side single-addrese im- 
age approach known as network address trans lation, as 
described in greater detail in E. Anderson, D. Patterson 
and E. Brower, 'The Maglcrouter, an Application of Fast 
Packet Interpostig,' Symposium on Operating Systems 
Design and Implementation, OSDI. 1996, <httpy/www 
cs.borkoloy.odu/^oandors/rnagicroutor/ osdi9S-mr 
submis6ion.ps>, and Cisco Local Director, <http://www 
cisco.comAvarp/pubhc/7Sl/lodir/index.html> ( which are 
incorporated by reference herein. As in the TCP rouier 
approach of FIG. 2. thectient 12 uses the router address 
RA as a destination IP address n a client request, and 
the rouier 32 dispatches the requ est to a s elected server 
1 4-k by changing the destination IP address of each in- 
coming request packet from the router address RA to 
the address Sk of selected server 14-k. However, in the 
network address translation approach, the source IP ad- 
dresses In the reply packets from the selected server 
14-k.are changed not by server 14-k as in FIG. 2, but 
are instead changed by the router 32. The reply packet 
flow indicated by a dashed line h FIG. 2 thus passes 
from server 14-k to client 12 via router 32. 

Compared to the TCP routor approach or FIG 2 
network address translation has the advantage of server 



transparency. That is. no specific changes to the kernel 
code ol the servers are required to implement (he tech- 
nique. However, both (he TCP rouier and network ad- 
dress Iranslation approaches require tha( the destrna- 
* ton address in a request packet header be changed to 
a server address so that the server can accept the r 9 . 
quesL These approaches also require that the source 
address n a reply packet headar be changedto the rout- 
er address so that the client can accept the reply. These 
" changes introduce additional processing overhead and 
unduly complicate the packet delivery process. In addi- 
tion, because of (he address changes, the ebove.de. 
scribed eingle-address image approaches may not be 
suitable for use with protocols that utilize IP addresses 
" within an application, such as that described in K 
Egevang and P. Francis. The IP Network Address 
Translator.' Network Working Group, RFC 1631 <htt D - 
//www.safety.neV rfc1631.0rt», which is Hcorporeted by 
reference herein. Furthermore, in both the TCP router 
and network address translation approaches, the router 
32 needs to store an IP address mapping for every IP 
connection. Upon receiving an incoming packet associ- 
ated with an existing TCP connection, the router has to 
search through al of the mappings to determine which 
« server the packet should be forwarded to. The router 
rtsoH may therefore become a bottleneck under heavy 
load conditions, necessitating the use of a more com- 
plex hardware design, as In the above^ited Cisco Local 
Director. 

30 It is therefore apparent that a need exist* lor im- 
proved techniques for hosting a network service on a 
cluster of servers while presenting a single -address inv 
age to (he clients, without the problems associated with 
^ (he above-described conventional approaches. 

Summary of the Invention 



The present invention provides methods and appa- 
ratus for hosting a network service on a cluster ol serv- 
*o ers. An of (he servers in a servor cluster conligured in 
accordance- with the invontion may bo dosignatod by a 
single cluster address which is assigned as a secondary 
address to each server. All client requests tor a web she 
or other network service associated with the cluster ad- 
« dress are sent to the server cluster, and a dispatching 
mechanism is used to ensure that each client request is 
processed by onV one server in the cluster. The dis- 
patching may be configured to operate without increas- 
ing the number of TCP/IP connections required for each 
so client request. The invention evenly distributes the client 
request load among the various servers of the cluster, 
masks the failure of any server or servers of the cluster 
by distributing client requests to the remaning servers 
without bringing down the service, and permits addition- 
's al servers to be added to the cluster without bringing 
down the service. Although wefl-suiled lor use ri hosting 
web site services, the techniques of the present inven- 
tion may also be used to support a wide variety of other 
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server applications. 

in an exemplary embodiment ol the invention, a net* 
work e en/ice is hosted by a server cluster in which each 
server includes a primary IP address and a secondary 
IP address. A common cluster address is assigned as s 
the secondary IP address for each ot the servers. The 
cluster address may be an IP address which does not 
correspond to a primary IP address ol any o! the servers. 
In UNIX-based servers, the cluster address may be as* 
signed as the secondary address tor a given server us- -0 
ing an jfcontig alias option. II a given sen/or includes 
multiple network Interface cards, the cluster address 
may be assigned to one of the network interlace cards 
using a UNIX ftconfig command without the alias option, 
or other similar technique. A router is coupled to a local *J 
network of the server cluster and Is also coupled via the 
Internet to a client. The router receives client requests 
from the Internet, and uses a dispatching technique to 
direct cliont re que sis having the duster address as a 
destination. The client requests are dispatched such so 
that each of the requests is processed by only one of 
the servers In the cluster. The dispatching (unction may 
be based on the result of applying a hash function lo an 
IP address of the given client. A suitable hash (unction 
may be determined using an analysis ol a dislribution of 25 
client IP addresses in an access tog associated with one 
or more o1 the servers, tn the event that a server has 
(ailed, the hash functbn may be reapplied to the client 
IP address to identify another server. 

Two illustrative dispatching techniques lor providing 30 
a single- address image for a server cluster in accord* 
ance with the invention indudo routing -based dispatch- 
ing and broadcast-based dispatching. In the routing- 
based technique, a dispatcher is coupled to the router 
and to a local network of the carver cluster. The router 35 
directs client requests having the cluster address to the 
dispatcher, and the dispatcher selects a particular one 
ol the servers to process a given client request based 
on the result of applying a hash (unction to the client 
address. In the broadcast-based technique, the router *o 
broadcasts client requests having (ho cluster address to 
each of the servers over the local network of the server 
cluster. Each of the servers implements a filtering rou- 
tine to ensure that each cHent request is processed by 
only one of the servers. The filtering routine may involve 4 * 
applying a hash function to the client IP address asso- 
ciated with a given client request, and comparing the 
result to a server identifier to determine whether that 
server should process the dient request. 

The techniques ol the present invention provide fast so 
dispatching and can be implemented with reduced cost 
and complexity. The techniques are suitable lor use in 
TCP/IP networks as well as networks based on a variety 
ot other standards and protocols. Unlike the convention- 
al single-address image approaches, the present inven* ss 
lion does not require that a destination address in a re- 
quest packet header bo changed to a server address so 
that the server can accept the request, or that a source 



address in a reply packet header be changed to the rout- 
er address so that the client can accept the reply. In ad- 
dition, the router need not store an IP address mapping 
(or every IP connection, nor is it required to search 
through such a mapping to determine which server a 
packet should be forwarded to. The router itself will 
therefore not become a bottleneck under heavy load 
conditions, and special router hardware designs are not 
required. These and other features and advantages of 
the present invention will become more apparent from 
the accompanying drawings and the loltowing detailed 
descriplicn. 

Brief Description of the Drawings 

FIG. 1 1s a block diagram illustrating a conventional 
cSent-server interconnection in accordance with the 
TCP/IP standard; 

FIG. 2 illustrates a prior art TCP router technique 
for hosting a network service on a cluster of servers; 
FIG. 3 illustrates a prior art network address trans- 
lation technique lor hosting a network servico on a 
cluster of servers; 

FIG. 4 illustrates a technique tor hosting a network 
service on a cluster of servers using rout ing -based 
dispatching in accordance with an exemplary em- 
bodiment ol the invention; and 
FIG. 5 illustrates a technique (or hosting a network 
service on a cluster ot servers using broadcast- 
based dispatching in accordance with another ex- 
emplary embodiment ot the invention. 

Detailed Description of the Invention 

The present invention will be illustrated below in 
conjunction with exemplary client/ server connections 
established over the Internet to a server cluster using 
the Transmission Control Protocol/Internet Protocol 
(TCP/IP) standard. It should be understood, however, 
that the invention is not limited to use with any particular 
type ol networkor network communication protocol. The 
disclosed techniques are suitable for use with a wide 
variety ol other networks andjsrotpcols. The term "serv- 
er cluster* as used herein refers to a group or set ol serv- 
ers interconnected or otherwise configured lo host a not- 
work service. The terms 'cluster address* and 'single- 
address image* refer generally to an address associat- 
ed with a group of servers configured to support a net- 
work service or services. A 'ghost IP address* is one 
type ol cluster address in the form of an IP address 
which is not used as a primary address for any server 
ol a given server cluster. The term "network service" is 
intended to include web sites, Internet sites and data 
delivery services, as well as any other data transler 
•mechanism accessible by a client over a network. The 
term 'client request* refers to a communication from a 
cliont which initiates the network service. A given client 
request may include muttiole packets or only a single 
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packet, depending on the nature of the request. 

The present Invention provides an improved single- 
address image approach to distributing client requests 
to sen/are of a server cluster. In a preferred embodi- 
ment, the Invention allows ad servers o! a server cluster 
to share a single common IP address as a secondary 
address. The secondary address is also referred to 
herein as a cluster address, and may be established us- 
ing an Hoofing alias option available on most UNIX- 
baeed systems, or similar techniques available on other 
systems. The cluster address may be publicized to cli- 
ents using the above-noted Domain Name Service 
(DNS) which translates domain namos associated with 
Uniform Resource Locators (URU) to IP addresses. All 
client requests to be directed to a service hosted by the 
server duster are sent to the single cluster address, and 
dispatched to a selected one ol the servers using rout- 
ing-based or broadcast-based dispatching techniques 
to be described In greater detail below. Onco a server 
is selected, future request packets associated with the 
same client request may be directed to the same server. 
All other communications wtthfn the server cluster may 
utilize primary IP addresses of the servers. 

The above-noted ffconfig arias option is typically 
used to allow a single server to serve mors than one 
domain name. For example, the ifconfig aVas option al- 
lows a single server to attach multiple IP addresses, and 
thus multiple domain names, to a single network Inter- 
face, as described In 'Two Servers, One Interface' <ht- 
tpV/wft¥w.thesphere.ccnV*-dlp/TwoServers/> ( which is 
incorporated by reference herein. Client requests direct- 
ed to any of the multiple domain names can then be 
serviced by the same server. The server determines 
which domain name a given request Is associated with 
by examining the destination address in the requost 
packet. The present invention utilizes the ifconftg alias 
option to allow two servers to share the same IP ad- 
dross. Norma ly. two servers cannot share the samo IP 
address because such an arrangemeni would cause 
any packet destined for the shared address to be ac- 
cepted and responded to by both sorvors, contusing the 
client and possibly leading to a connection reset. There- 
lore, before a server is permitted to attach a new IP ad- 
dress to its network interlace, a check may be made to 
ensure that no other server on the same focal area net* 
work (LAN) is using that IP address. It a duplicate ad- 
dress Is found, both servers ere informed and warnings 
are issued. The routing-based or broadcast-based dis- 
patching of the present invention ensures that every 
packet Is processed by only one sen/er ol the ciuster. 
such that the above-noted warnings do not create a 
problem. 

An alternative technique for assigning a secondary 
address to a given server of a server cluster in accord- 
ance with the invention involves configuring the given 
server to include multiple network interface cards such 
thai a different address can be assigned to each of tho 
network interface cards. For example, in a UNIX-based 



system, conventional ifconfig commands may be used, 
without the above-described alias option, to assign a pri- 
mary IP address to one of the network interface cards 
and a secondary IP address to another of the network 
i interface cards. The secondary IP address is also as- 
signed as a secondary IP address to the remaining serv- 
era in the cluster, and used as a cluster address for di- 
recting client requests to the cluster. 

The exemplary embodiments of the present inven- 
10 ijon to be described below utilize dispatching techniques 
in which servers are selected based on a hash value ol 
the client IP address. The hash value may be generated 
by applying a hash function to the cliont IP address, or 
by apprying another suitable function to generate a hash 
value from the client IP address. For example, given N 
sorvors and a packet Irom a client having a cliont ad- 
dress CA, a dispatching function in accordance with the 
invention may compute a hash value k as CA mod (N- 
I) and soloct server Vto process tho packet. This en- 
sures that all request or reply packets of the same TCP/ 
IP connection are directed to the same server in the 
server cluster. A suitable hash function may be deter- 
mined by analyzing a distribution of client IP addresses 
in actual access logs associated with the servers such 
2S that client requests are approximately evenly distributed 
to all servers. When a eerver in the cluster fails, the sub- 
set of clients assigned to that server wiil not be able to 
conned to il. The present invention addresses this po- 
tential problem by dynamically modifying the dispatch- 
50 ing Junction upon detection of a server failure. If the hash 
value of a given client IP address maps to the failed serv- 
er, tho client IP address is rehashed to map to a non- 
failed server, and the connect tons of the remaining cli- 
ents are not affected by the failure. 
** FIG. 4 illustrates a routing-based dispatching tech- 
nique in accordance with the present invention. Solid 
lines indicate network connections, while dashed lines 
show the path of an oxemplary client request and the 
corresponding reply. A client 52 sends a client request 
*o to a server cluster 54 including N servers 54-i, i = 1 , 2, . 
N having IP addresses St. S2. ... SN and interconnect- 
ed by an Ethernet or other type of LAN 56. The client 
request is formulated in acco/dance with the above de- 
scribed HTTP protocol, and may include a URL with a 
*s domain name associated wilh a web site or other net- 
work service hosted by the server cluster 54. The client 
accesses a DNS to determine an fP address for the do- 
main name of the service, and then uses the IP address 
to establish a TCP/IP connection for communicating 
s° with one of the servers S4-i of the server ciuster 54. in 
accordance with the invention, a 'ghost* IP address is 
publicized to the DNS as a cluster address for the server 
cluster 54. The ghost IP address is selected such that 
none of the servers 54-iol cluster 54 has that IP address 
ss as ils primary address. Therefore, any request packets 
directed to the ghost IP address are associated with cli- 
ont roquosts for the servico of the singlo-addross image 
cluster 54. The use oi the ghost IP address thus distin- 
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guishes a network service hosted by Ihe cluster from 
activities of the servers 54-i which utilize the primary 
server addresses, and prevents interference with these 
primary address activities. 

The client 52 uses the ghost IP address as a cluster 
address lor directing its request to the server cluster 54. 
The request is directed over internet 60 to a router 62 
having an IP address RA. The router 62 includes a rout- 
ing table having an entry or record directing any Incom- 
ing request packets having the ghost IP address to a 
dispatcher 64 connected to the LAN 56. The dispatcher 
64 Includes an operating system configured to run in a 
router mode, using a routing algorithm which performs 
the dispatching described herein. In alternative embod- 
iments, the functions of the dispatcher 64 could be in* 
c expo rated into lha router 62 in order to provide addi- 
tional efficiency improvements. Each of the servers 54-i 
ot the cluster 54 utilizes the above-described ifconfig ali- 
as option to sot tho ghost IP address as their secondary 
address. As noted above, this technique for setting a 
secondary add resslor each ol the servers 54-i generally 
does not require any alteration of the kernel codo run- 
ning on the servers. In alternative embodiments, one or 
more of the servers 54-i may be configured to include 
multiple network interlace cards, as previously noted, 
such that a different address can be assigned to each 
cl the network interface cards of a given server using a 
UNIX ffconfig command or other similar technique. 

The router 62 routes any packets having the ghost 
IP address to the dispatcher 64 in accordance with the 
above-noted routing table record. The dispatcher 64 
then applies a hash function to the client IP address in 
a given request packet to determine which of the servers 
54-1 the given packet should be routed to. In the example 
illustrated in FIG. 4, the dispatcher 64 applies a hash 
function to the IP address ol client 52 and determines 
that the corresponding request packet should be routed 
to server 54-2 at IP address S2. The dispatcher 64 then 
routes the request packet to the server 54-2 over LAN 
56. as indicated by the dashed line, using the primary 
address S2 of server 54-2 to distinguish II from tho other 
servers of cluster 54. After the network interlace of serv- 
er 54-2 accepts the packet, all higher level processiog 
may be based on the ghost IP address because that is 
the destination address in Ihe packet IP header and pos- 
sibly in the application -layer packet contents. After 
processing the request, the server 54-2 replies directly 
to the client 52 via router 62 over the established TCP/ 
IP connection, using the ghost IP address, and without 
passing through the dispatcher 64. 

M should be noted that when a request packet des- 
tined for the ghost IP address is received by the network 
Interface of dispatcher 64 and placed back onto the 
same network interlace for delivery to one ol tho servers 
54-i over LAN 56. ft may cause an Internet control mes- 
sage prolocol (1CMP) host redirect message to be sent 
to tho router 62. This ICMP message is dosignod to di- 
rect the router 62 to updale its routing table such that 



any future packets having the ghost IP address can by- 
pass the dispatcher 64 and go directly to the destination 
server, as described in greater detail In W.R. Si evens, 
TCP/IP Illustrated, Vol. 1, Ch. 6, pp. 69-83, which is in- 
s corporal ed by relerence herein. However, this effect is 
undesirable In the routing technique ol FIG. 4 because 
the dispatcher 64 performs the server selection process 
as previously described. It therefore may be necessary 
to suppress the ICMP host redirect message for the 
io ghost IP address by. for example, removing or altering 
the corresponding operating system code in the dis- 
patcher. In the above-mentioned alternative embodi- 
ments in which the dispatching I unci bo is implemented 
wrthin the router 62. the tCMP redirect message is not 
>* generated and therefore need not be suppressed. An- 
other potential problem may arise when a reply packet 
is sent back to the client 52 from the selected server 
54-2 with the ghost IP address, In that it may cause the 
rouior 62 to associate, in its Address Resolution Proio- 
20 col (ARP) cache, the ghost IP address with the LAN ad- 
dress of the selected server. The operation ot the ARP 
cache is described in greater detail in W.R. Stevens. 
TCP/IP Illustrated. Vol. 1 1 Chs. 4 and 5, pp. 53-68, which 
is incorporated by reference herein. The illustrative em* 
2S oodiment ct FIG. 4 avoids this problem by automat icalry 
routing the request packets lo the dispatcher 64. and 
(hen ctspatching based on the server primary IP ad- 
dress, such thai the router ARP cache Is not used. 
FIG. 5 illustrates a broadcast-based dispatching 
so technque in accordance with the present invention. 
Again, so W lines indicate' network connections, while 
dashed lines show tho path of an exemplary dienl re- 
quest and the corresponding reply. As in the RG. 4 rout- 
ing-based embodiment, client 52 sends a client request 
3S to server cluster 54 including N servers 54-i. i = 1 . 2. ... 
N connected to LAN 56 and having IP addresses Si, 
S2, ... SN. The client 52 uses the above -described ghost 
address as a cluster address for directing Its request to 
the server cluster 54. The request is directed over Inter- 
*o net 60 to a router 70 having en IP address RA. The rout- 
er 70 broadcasts any Incoming request packets having 
the ghost IP address to the LAN 56 interconnecting the 
servers 54-i of the server cluster. 54, such that the re- 
quest packet is received by each ol the servers 54-i. 
<s Each ol Ihe servers 54-i ol Ihe duster 54 imple- 
ments a littering routine in order to ensure that only one 
of the servers 54-i processes a given client request. The 
filtering routine may be added to a device driver cl each 
ol the servers 54-i. In an exemplary implementation, 
so each ot the servers 54 -I is assigned a unique identifica- 
tion (10} number. The filtering routine ol a given server 
54-i computes a hash value of Ihe client IP address and 
compares il to the 10 number of the given server. If the 
hash valuo and ihe 10 number do nol match, the filtering 
55 routine ol the given server rejects the packet, if the hash 
value and the ID number do match, the given server ac- 
costs and processes tho packet as if a had received the 
packet through a conventional I P routing mechanism. In 
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the illustrative example of FIG. 5, a packet associated 
with request from client 52 is broadcast by the router 70 
to each of the servers 54-1 of the server cluster 54 over 
the LAN 56 as previously noted. The filtering routine ol 
server 54*2 generates a hash value or the client IP ad- 
dress which matches the unique ID number associated 
wan server 1 4-2. and server 1 4*2 therefore accepts and 
processes the packet The filtering routines of the N-1 
other servers 54-I each Indicate no match between the 
client IP address and the corresponding server ID 
number, and therefore discard the broadcast packet. 
The reply packets are sent back to the client 52 via rout- 
er 70. as Indicated by the dashed lines, using the ghost 
IP address. 

The broadcast -based dispatching technique of RG. 
S may be Implemented using a permanent ARP entry 
within the router 70. to associate the ghost IP address 
with the Ethernet or other local network broadcast ad- 
dress associated with LAN 56 of the cluster 54. A po- 
tential problem is that any reply packet from a selected 
server appears to be coming from the ghost I P address, 
and may therefore cause the router 70 to overwrite the 
entry in its ARP cache such that the ghost IP address is 
associated with the LAN address ol the sefected server. 
This potential probJem may be addressed by setting up 
a routing table entry in the router 70 to direct all packets 
having a ghost IP destination address to a second ghost 
IP address which is a legal subnet address in the LAN 
56 of the server cluster 54 but Is not used by any server. 
In addition, an entry is inserted in the ARP cache ol the 
router 70 to associate the second ghost IP address with 
tho broadcast address of the LAN 56. When tho roulor 
70 routes a packet lo the second ghost IP address, it 
will then actually broadcast the packet to each of the 
servers 54-I of the cluster 54. Since no reply packet is 
sent from the second ghost IP address, the correspond- 
ing entry of the router ARP cache will remain un- 
changed. Another potential problem is that some oper- 
ating systems, such as the NetBSD operating system, 
do not allow a TCP packet to be processed if it is re- 
coived (rom a broadcast address. This poientlal problem 
may be avoided by a suitable modification to the broad- 
cast address in the LAN packet header attached to the 
packet. 

The rouling-based and broadcast-based dispatch- 
ing techniques descnbed in conjunction with FIGS. 4 
and 5 above have been Implemented on a cluster ol Sun 
SPARC workstations. The NetBSD operating system, 
as described in NetBSD Project, <httpVwww. NetBSD. 
org>, was used to provide any needed kernelcode mod- 
ifications. The dispatching overhead associated with 
both techniques is minimal because the packet dis- 
patching is baaed on simple IP address hashing, without 
the need lor storing or searching any address -mapping 
information, in the routing-based dispatchhg technique, 
the additional routing step in the dispatcher 64 typically 
adds a delay of about t to 2 msocs to tho TCP round- 
trip time of each incoming request packel. A study in W 



R. Stevens, TCP/IP Illustrated. Volume 3. pp. 165-186. 
which is incorporated by reference herein, indicates that 
the median TCP round-trip time is 187 msecs. The ad- 
ditional delay attributable to the routir.g-based dispalch- 
s ing is therefore negligible. Although the additional rout- 
ing step for every request packet sent to the ghost IP 
address may increase the traffic in the LAN of the server 
cluster, the size ol a request in many inportant applica- 
tions Is typicafty much smaller than that of the corre- 
'0 sponding response, which Is delivered direclly to the cli- 
ent without the additional routing. In the broadcast- 
based dispatching technique, the broadcasting of each 
incomhg roquest packet on tho LAN of iho server clus- 
ter does not substantially increase network traffic AJ- 
« though a hash value is computed for each incoming 
packet having the ghost IP destination address, which 
increases the CPU lead of each server, this additional 
computation overhead is negligible relative to the corre- 
sponding communication delay. 
20 Both the routing^based and broadcast-based dis- 
patching techniques of the present invention are scala- 
ble to support relatively large numbers of servers. Al- 
though the dispatcher in the routing-based technique 
could present a potential bottleneck ri certain applica- 
« lions, a study in the above-crted D M. Dias et el. refer- 
once indicates thai a single dispatcher can support up 
to 75 server nodes, which is sufficient support tor many 
practical systems. The number of servers supported 
maybe even higher with the ptBzent invention given that 
30 the routing-based dispatching functions described here- 
in are generalry simpler than those In the D.M. Dias et 
at. roforence. It shouldalso bo noted that additional scal- 
ability can be obtained by combining the routing-based 
dispatching of the present invention with a DNS round- 
* robin technique. For example, a DNS server may bo 
used to map a domain name to one of a number of dif- 
ferent ghost IP addresses belonging to different server 
clustors using a round-robin technique In tho broad- 
cast-based dispatching technique, there is no polenlial 
«o dispatching bottleneck, although the device drivers or 
othor portions of tho servers may need to bo modified 
to provide Ihe above -described filtering routines. 

The routing-based and t?roadcast-based dispatch- 
ing of the present invention can also provide toad bal- 
*s ancing and failure handling capabilities. For example, 
given N servers and a packel from client address Ca! 
the above-described routing-based dispatching function 
may compute a hash value k as CA mod(N- 1) and select 
server k lo process the packet. More sophisticated dis- 
« patching functions can also be used, and may involve 
analyzing the actual service access log lo provide more 
effective load balancing In order to detect failures, each 
server may be monitored by a watchdog daemon such 
as the wafend daemon described in greater detail in Y. 
« Huang and C. Kinlala, 'Software Implemented Fault Tol- 
erance: Technologies and experience.' Proceedings of 
tho 23 rd international Symposium on Fault -Tolerant 
Computing - FTCS. Toulouse, France, pp. 2-9, June 
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1993, which is tncofporatod by reference herein. When 
a server falls, the corresponding watchd daemon initi- 
ates a change of the dispatching function to mask the 
failure and rebalance the load. A system call interface 
may be implemented to allow the dispatching function s 
lobe changed while the servers remain on-line. In foul- 
ing- based dispatching, the watchd daemon may notify 
the dispatcher to change the dispatching (unction, while 
in broadcast-based dispatching, aQ servers may be no- 
tified to modify their filtering routines. For example, if a »o 
server k fails, the new dispatching function may check 
10 see if the hash value CA mod N equals k. It it does, 
a new hash value / = CA mod (N-1 ) is computed. If > is 
less than k, the packet goes to server / Otherwise, the 
packet goes to server/+f. This technique does not affect 
the clients of non-failed servers, reassigns the clionts of 
the failed server evenly to the remaining servers, and 
can be readily extended to handle muliipte server fail- 
ures. Additional servers can be added to iho cluster 
without bringing down the service by changing the dis- so 
patching function from CA mod NxoCA mod (irV+1/ 

In routing-based dispatching, tho dispatcher may 
become a single point of failure, and therefore should 
also be monitored by a wa tend da em on or other suitable 
failure monitoring mechanism. Upon detecting a failure. 2i 
the watchd daemon may trigger a transfer ol the dis- 
patching function 1rom the primary dispatcher to a back- 
up dispatcher, and then direct the router to change the 
entry in its routing table such that future incoming re- 
quest packets are routed to the backup dispatcher, so 
Since no mappingtable is maintained by the primary dis- 
patcher, this approach is substantially stateless. Proper 
routing may be ensured by simply utilizing consistent 
routing functions in the primary and backup dispatchers, 
without the substantial additional costs associated with 
mapping-based approaches. 

The use ol the ifconfig alias option or other similar 
i echnlquo to provide a singto-eddrcss imago for a server 
cluster provides a number of advantages over the con- 
ventional techniques described previously. For exam- 
ple. I: avoids the need to chango the destination address 
in a request packet header so that a particular server 
can accept the request, and the need to change tbe 
source address fn a reply packet header to the cluster 
address so that the client can accept the reply. With the 
single -address image approacn ol the present inven- 
tion, all servers can accept and respond to packets hav- 
ing the cluster address, so thai the addresses in the re- 
quest and reply packet headers do not need to be mod- 
iliod. Since the single-image approach of the present in- 
vention does not require alternation of the packet ad- 
dresses, it is suitable for use with a wide variety ol pro- 
tocols, deluding those protocols which utilize IP ad- 
dresses within an application program. In addition, the 
single-address image approach of the present invention 
does not require a router to store or to search through 
a potentially largo number of IP addross mappings In 
order to determine which cluster server should receive 



a request packet. The invention thus effectively re- 
moves the possibility that the router may become a bot- 
tleneck under heavy load conditions. 

The above -de scribed embodiments of the invention 
are intendedtobe illustrative only, Numerous alternative 
embodiments may be devised by those skilled in the art 
without departing from the scope of the foflowing claims. 



Claims 

1 . A method of routing client requests to a plurality of 
servers configured to support a network, service 
over a communication network, each of the servers 
having a primary address, the method comprising 
the steps of: 

assigning a common address as a secondary 
address for each of the plurality of servers; and 
processing client requests directed to the com- 
mon address such that each of the requests is 
processed by a particular one of the plurality or 
servers. 

2. The method of claim V wherein the network utilizes 
a TCP/IP protocol and the primary and secondary 
addresses are primary and secondary IP address- 
es, respectively. 

3. The method of claim 2 wherein the common ad- 
dress is an IP address which does not correspond 
to a primary IP address of any of the plurality of serv- 
ers. 

35 4. The method ol any of the preceding claims wherein 
at least one of the plurality of servers is a UNIX- 
based server including multiple network interface 
cards, and the assigning step Includes assigning 
the common address for the at least one servor us- 
mo ing an ifconfig command.. 

5. The method of any of claims t to 3 wherein the plu- 
rality ot servers are UNIX-based servers, and the 
assigning step includes assigning the common ad- 

*s dress utilizing an ifconfig alias option for ai least a 
subset of the plurality of servers. 

6. The method of any of the preceding claims wherein 
the processing step includes the step of dispatching 

so a requesi of a given client to one of the plurality of 
servers based on application ot a hash lunciion to 
an IP address of the given cltont 

7. Tho method ol claim 6 wherein the hash (unction is 
55 determined based on an analysis of a distribution 

ot client IP addresses in an access log associated 
with one or more ol the servers. 
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8. The method of claim 6 wherein Ihe dispalching step 
include* reapplying the hash (unction to tho cliont 
IP address to Identify another servers If a server 
identified as a results of a previous application of 
tho hash function has failed. 5 

9. The method of any of claims 1 to S wherein the 
processing slop includes the steps of: 

routing client requests directed to the common '0 
address to a dispatcher connected to a local 
network associated with the plurality of servers: 
and 

selecting a particular one of the servers to proc- 
ess a given client request based on application is 
ot a hash function to a corresponding client ad- 
dress in the dispatcher. 

10. The method of any ol claims 1 to 5 wherein tho 
processing step includes the steps of: 20 



11. The method of claim 10 wherein the implementing so 
step Includes the steps of: 

applying a hash function to a client IP address 
associated with the given dient request; and 
comparing the result of the applying slop to an « 
identifier ot a particular server to determine 
whether that server should process the given 
client request. 

12. An apparatus for routing client requests to a plurality *c 
of servers configured to support a network sorvico 
over a communication network, each of the servers 
having a primary address, the apparatus compris- 
ing: 

4S 

means for assigning a common address as a 
secondary address for each ot the plurality of 
servers; and 

means lor processing client requests directed 
to the common address such that each ol the so 
requests is processed by a particular one of (he 
plurality ol servers. 

13. The apparatus of claim 12 wherein the processing 
means is operative to dispatch a request of a given ss 
client to one of the plurality of servers based on ap- 
plication of a hash function to an IP addross of the 
given client. 



14. The apparatus of claim 13 wherein tho hash func- 
tion is dolorminod based on an analysis of a distri* 
b ution 01 client IP addresses In an access log asso- 
ciated with one or more of the servers. 

15. The apparatus of claim 13 wherein ihe processing 
means is further operative to reapply the hash func- 
tion to the client fP address to identify another serv- 
er if a server identified as a result ol a prevbus ap- 
plication of ihe hash (unction has failed. 

IS. The apparatus of any of claims 1 2 to 1 5 wherein the 
processing moans further includos a dispatcher 
connected to a tocaj network associated with the 
plurality of servers, wherein the dispatcher is oper- 
ative to receive cliont requests direciod to tho com- 
mon address, and to select a particular one of the 
servers to process a given client request based on 
application et a hash t unction to a corresponding 
client address. 

17. The apparatus of any of claims 12 to 15 wherein the 
processing means further includes: 

means for broadcasting a given client request 
directed to the common address to each of the 
plurality of servers over a local networtc associ- 
ated with the servers; and 
means lor filtering the given client request in 
each of the pluraJity of servers so that the given 
dient request a processed by only one of the 
servers. 

13. The apparatus of claim 17 wherein the filtering 
means is operative to apply a hash function to a cli- 
ent IP address associated with the given client re- 
quest, and to compare the result ol the applying 
step to an idoniifior of a particular sorvor to dotor- 
ming whethor that sorver should process the given 
cliont request. 

19. An apparatus for routing client requests for a net- 
work service over a communication network, the 
apparatus ccmpnslng: 

. a plurality ol servers configured to support the 
network service, each of the servers having a 
primary address and a secondary address, 
wherein a common address is assigned as Ihe 
secondary address lor each of the plurality of 
servers: and 

a rouler coupled to the servers and operative 
to route clieni requests directed to the common 
address such that each 0/ the requests is proc- 
essed by a particular one of the plurality ol serv- 
ers. 

20. The apparatus ol claim 1 9 wherein the router is fur- 



broadcasting a given client request directed to 
the common address 10 each of the plurality o* 
servers over a local network associated- with 
the servers: and 2$ 
implementing a filtering routine in each cf the 
plurality of servers so that the given client re- 
quest Is processed by onfy one of the servers. 
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ther operative to route client requests such that a 
request of a given client is routed to one ol the plu- 
rality of servers based on application of a hash func- 
tion to an IP address of Ihe given client. 

21. The apparatus of claim 20 wherein ihe hash func- 
tion is determined based on an analysis of a distri- 
bution ol client IP addresses in an access log asso- 
ciated with one or more of the servers. 

22. The apparatus of claim 20 wherein the hash func- 
tion is reapplied to the client IP address to identify 
another server if a server identified as a result of a 
previous application of the hash function has failed. 

23. The apparatus of any of claims 1 9 to 22 funhor in- 
cluding a dispatcher coupled to the rcuier and to a 
local network associated wiih the plurality of serv- 
ers, such that tho router directs client requests hav- 
ing the common address to the dispatcher, and the 
dispatcher selects a particular one of the servers to 
p roc oss a grvon client request based on application 
of a hash function to a corresponding client ad- 
dress. 

24. The apparatus of any of claims 1 9 to 22 wherein the 
router is further operative to broadcast a given client 
request directed to the common address to each of 
the plurality of servers over a local network associ- 
ated with the servers, and further wherein each of 
the servers Implements a filtering routine so that the 
given client request is processed by only one of tho 
servers. 

25. The apparatus of claim 24 wherein the filtering rou- 
tine involves applying a hash function to a ciiont IP 
address associated with the given client request. - 
and comparing tho resuft loan identifier of a partic- 
ular server to determine whether that server should 
process tho given client request 

26. The apparatus of any of claims 1 2 to 25 wherein trie 
network utilizes a TCP/IP protocol and the primary 
and secondary addresses are primary and second- 
ary IP addresses, respectively. 

27. The apparatus of claim 25 wherein (he common ad- 
dress is an IP address which does not correspond 
toa primary IP address of any of Tho plurality ol serv- 
ers. 

28. The apparatus ot any of claims 1 2 to 27 wherein at 
least one ot the plurality ot servers is a UNIX-based 
server including multiple network interlace cards, 
and the common address is assigned for the at least 
one server using an ifconfig command. 

29. Tho apparatus of any of claims t2to27 wnoroinno 



pluratrtyof servers are UNIX-based servers, and the 
common address is assigned as the secondary ad- 
dress of the plurality ol servers by utilizing an ifcon- 
fig alias option for at least a subset of the plurality 
s of servers. 
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